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CROSS-REFERENCE TO RELATED APPLICATIONS 
[0001] This application claims priority to U.S. provisional application Serial No. 
60/237,666, filed October 3, 2000. This co-pending application is incorporated herein by 
reference in its entirety. 

TECHNICAL FIELD 

[0002] The invention relates generally to automated supplier selection. More specifically, 
the invention relates to an apparatus and methods for utilizing a supplier-rating matrix to 
obtain information about those services for which a given supplier is best suited. 

BACKGROUND 

[0003] Companies engaged in commerce have often looked to third party rating systems 
for guidance when electing a potential business partner. Firms such as Moodys and Standard 
& Poor rate a firm's debt offerings, and Dun and Bradstreet (D&B) provides ratings about a 
firm's creditworthiness. With the rise of a global economy, firms have more choices for 
partners and an increased need for objective third party ratings. 

[0004] Internet commerce companies such as OpenRatings and eBay provide ratings 
based on feedback from prior customers. More elaborate systems have been developed to rate 
distributors and retailers in marketplaces for known products, such as those employed by 
Frictionless Commerce. (See, for example, Guttman, Robert H., "Merchant Differentiation 
through Integrative Negotiation in Agent-minded Electronic Commerce", MS Thesis, MIT 
Department of Media Arts and Sciences, May 7, 1999). 



[0005] All of these rating systems help firms understand the prospective business partner's 
ability to please its customers. The weakness of such systems is that they do not take into 
account the idiosyncrasies of the service for which the party was rated. For example, if a 
party has received numerous positive ratings as a seller and shipper of record albums, that 
rating has little relevance if you are considering shipping services for furniture. 

SUMMARY OF THE INVENTION 
[0006] The invention fills the need for unbiased rating information that continues to grow 
as companies expand their reach for trading partners. The invention features a supplier-rating 
matrix for rating and predicting future performance of service providers, providing valuable 
insight for buyers looking to purchase specialized services in a marketplace. 
[0007] The invention uses customer satisfaction ratings, mapped against the attributes of 
the job or service performed to build a task-specific supplier-rating matrix. The supplier- 
rating matrix can be used with proposed job attributes and desired performance metrics to 
predict a supplier's performance for a specific proposed job. A traditional rating system 
answers the question "Is this a good vendor?". The subject invention answers the question "Is 
this a good vendor for this particular job?". Furthermore, the invention features a self- 
learning function, i.e. the ratings become more accurate as job performance knowledge is 
accumulated. The invention can be implemented in various ways. For example, it can be 
implemented in hardware and/or software. The invention also can be provided on a web page 
on the Internet, through a client/server system, over an Intranet, on an Internet appliance 
and/or on a personal computer. The system maybe used as part of marketplace with 
numerous buyers and suppliers or for a single enterprise in which case the enterprise is the 
sole buyer with a plurality of suppliers. 
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[0008] The invention can be applied in any marketplace in which suppliers provide 
services to buyers. For example, the invention could be used in the markets of construction 
services, manufacturing services, engineering services, catering, auto repair, or tailoring. One 
specific example of the invention is in the purchase and supply of custom fabricated metal and 
plastic parts. In the target market, the buyers are typically original equipment manufacturers 
(OEMs) looking to outsource the production of certain custom designed parts or components. 
These parts are not readily available and must be custom built to the buyer's engineering 
specifications. The suppliers are typically job shops that specialize in one or more metal or 
plastic fabrication technologies. Buyers want to find the supplier with the equipment and 
expertise that best matches the requirements of the job. Jobs vary widely in terms of such 
factors as the processes used (machining, molding, casting, etc) desired production volumes 
(from one part to millions), required precision, speed of delivery, and level of quality. 
Furthermore, the buyer may have a different set of performance goals for each job. For 
example, on some jobs the speed of completion may be the most important performance 
metric, while on others the cost or quality level may be paramount. 
[0009] In one aspect, the invention relates to a method for creating a supplier-rating 
matrix for rating services of a supplier. The method includes defining a plurality of job 
attributes each including a plurality of sub-attributes, each sub-attribute representing a range 
of job attribute values and defining a job attribute vector, the job attribute vector including a 
plurality of dimensions each corresponding to a sub-attribute. The method further includes 
defining a plurality of supplier performance metrics and defining a supplier performance 
vector, the performance vector including a plurality of dimensions each corresponding to a 
performance metric. The method further includes defining a first initial value for the job 
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attribute vector, defining a second initial value for the performance vector and generating a 
supplier rating matrix for the supplier by mathematically combining the job attribute vector 
and the performance vector. 

[00010] In one embodiment, the job attribute vector is a first job attribute vector, the 
performance vector is a first performance vector and the supplier rating matrix is a first 
supplier rating. In this embodiment, the method further includes receiving data associated 
with a specific service supplied to a customer of the supplier and generating a second 
performance vector in response to the received data. The method further includes generating 
a second job attribute vector in response to the specific service, the second job attribute vector 
indicating which range of job attribute values are associated with the specific service, defining 
a weighting factor and generating a second supplier rating matrix for the supplier by 
mathematically combining the first supplier rating matrix, the weighting factor, the second job 
attribute vector and the second performance vector. 

[00011] In another embodiment, the method includes providing a second supplier- 
rating matrix associated with a second supplier and receiving data associated with a proposed 
job. The method further includes generating a second performance vector in response to the 
received data and generating a second job attribute vector in response to the received data, the 
second job attribute vector indicating which range of job attribute values are associated with 
the proposed job. The method further includes selecting the first or second supplier based on 
the first supplier matrix, the second supplier matrix, the second performance vector and the 
second job attribute vector. In another embodiment, the method further includes rating the 
supplier based on the supplier-rating matrix, the second performance vector and the second 
job attribute vector. 
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[00012] In yet another embodiment, the method farther includes defining the plurality of 
job attributes and the plurality of sub-attributes using a technical requirements specification of 
a customer of a supplier. In another embodiment, the method further includes defining the 
plurality of job attributes to include dimensional tolerance, turnaround time and quantity. In 
another embodiment, the method further includes defining the plurality of sub-attributes 
corresponding to the dimensional tolerance attribute to include a plurality of ranges of 
tolerance values, defining the plurality of sub-attributes corresponding to the quantity attribute 
to include a plurality of ranges of quantity values and defining the plurality of sub-attributes 
corresponding to the turnaround time attribute to include a plurality of ranges of times. 
[00013] In yet another embodiment, the method further includes defining the plurality of 
performance metrics to include speed, quality, cost and service. In another embodiment, the 
supplier-rating matrix includes a number of columns and a number of rows. In this 
embodiment, the step of generating further comprises multiplying the job attribute vector by 
the performance vector, resulting in the supplier rating matrix including the number of 
columns corresponding to the number of dimensions of the job attribute vector and the 
number of rows corresponding to the number of dimensions of the performance vector. 
[00014] In yet another embodiment, the step of generating the second supplier rating 
matrix further comprises multiplying the second job attribute vector by the second 
performance vector, thereby generating a third supplier rating matrix and multiplying the first 
supplier rating matrix by the weighting factor, thereby generating a fourth supplier-rating 
matrix. The method further includes multiplying the third supplier rating matrix by the 
difference of one minus the weighting factor, thereby generating a fifth supplier rating matrix 
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and adding the fourth supplier rating matrix to the fifth supplier rating matrix, thereby 
generating the second supplier rating matrix. 

[00015] In another aspect, the invention relates to a system for creating a supplier-rating 
matrix for rating services of a supplier. The system includes an interface module and a matrix 
generator module. The interface module is adapted to define a plurality of job attributes each 
including a plurality of sub-attributes, each sub-attribute representing a range of job attribute 
values and adapted to define a plurality of performance metrics. The matrix generator module 
is adapted to define a job attribute vector with an initial value, the job attribute vector 
including a plurality of dimensions each corresponding to a sub-attribute, adapted to define a 
performance vector with an initial value, the performance vector including a plurality of 
dimensions each corresponding to a performance metric, and adapted to generate a first 
supplier rating matrix for the supplier by mathematically combining the job attribute vector 
and the performance vector. 

[00016] In another embodiment, a hardware and/or software module is used to 
automatically construct the job attribute vector from the data available over a typical 
corporate network. For example, commercial information such as the quantity desired and 
turnaround time are available from an Enterprise Resource Planning (ERP) system (such as 
those available from companies such as SAP, ORACLE, and JD EDWARDS). Geometric 
attributes such as size, and tolerance level are available from a Computer Aided Design 
(CAD) system (such as PROENGINEER from PTC, SOLIDWORKS and SOLIDEDGE). 
Material specifications are available from a Product Data Management System (such as those 
available from MATIXONE, EIGNER + PARTNER, and PTC). This automatic construction 
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of the job attribute vector makes use of the system easier and provides the user with full 
benefit of their computerized product information. 

[00017] In another embodiment, the job attribute vector is a first job attribute vector, the 
performance vector is a first performance vector and the supplier rating matrix is a first 
supplier rating matrix. In this embodiment, the interface module is further adapted to receive 
data associated with a specific service supplied to a customer of the supplier. The matrix 
generator module is further adapted to define a weighting factor, adapted to generate a second 
performance vector in response to the received data, adapted to generate a second job attribute 
vector in response to the specific service that indicates which range of job attribute values are 
associated with the specific service and adapted to generate a second supplier rating matrix 
for the supplier by mathematically combining the first supplier rating matrix, the weighting 
factor, the second job attribute vector and the second performance vector. 
[00018] In another embodiment, the system includes a selector module. The matrix 
generator module is further adapted to provide a second supplier-rating matrix associated with 
a second supplier. The interface module is adapted to receive data associated with a proposed 
job. The selector module is adapted to generate a second performance vector in response to 
the received data, adapted to generate a second job attribute vector in response to the received 
data, the second job attribute vector indicating which range of job attribute values are 
associated with the proposed job and adapted to select the first or second supplier based on 
the first supplier matrix, the second supplier matrix, the second performance vector and the 
second job attribute vector. In yet another embodiment, the system includes a server in 
communication with a client via a network, wherein the server includes the interface module 
and the matrix generator module. 
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[00019] In another aspect, the invention relates to an article of manufacture having 
computer-readable program means for creating a supplier-rating matrix for rating services of a 
supplier. The article includes computer-readable program means for performing the steps of 
the methods as described above. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[00020] The above and further advantages of the invention may be better understood by 
referring to the following description taken in conjunction with the accompanying drawing, in 
which: 

FIGS, la and lb are block diagrams of illustrative systems for generating and 
maintaining the supplier-rating matrix and employing it to select a supplier; and 

FIG. 2 is a flow diagram of an illustrative process for generating and maintaining the 
supplier-rating matrix and employing it to select a supplier. 

DETAILED DESCRIPTION 
[00021] As illustrated in FIG. la, an embodiment of the invention can include a stand- 
alone computing device 10. The computing device 10 includes a matrix generator module 20, 
a selector module 30 and an interface module 40. The matrix generator module 20 and the 
selector module 30 can be implemented in hardware and/or software. The matrix generator 
module 20 is in electrical communication with the selector module 30 and the interface 
module 40. 

[00022] The computing device 10 can be any personal computer (e.g., 286, 386, 486, 
Pentium, Pentium II, IBM PC-compatible, etc), Macintosh computer, RISC Power PC, X- 
device, workstation, mini-computer, main-frame computer or other computing device adapted 
to generate and utilize a supplier-rating matrix to obtain information about supplier services. 
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The computing device 10 can include a display screen, a keyboard and an input device (e.g., a 
mouse), not shown. The optional interface module 40 is in electrical communication with the 
keyboard, the display and/or the mouse. Any one of a variety of operating system platforms 
can run on the computer device 10, including, for example, DOS, Windows 3.x, Windows 95, 
Windows 98, Windows NT 3.51, Windows NT 4.0, Windows CE, Macintosh, Java, or Unix. 
[00023] As illustrated in FIG. lb, in another embodiment, the computer system 1 0 can be 
connected to a communications network 50, which can be, for example, a local-area network 
(LAN) or a wide area network (WAN) such as the Internet or the World Wide Web. The 
computer device 10 can connect to the network 50 through a variety of connections including 
standard telephone lines, LAN or WAN links (e.g., Tl, T3, 56kb, X.25), broadband 
connections (ISDN, Frame Relay, ATM), or wireless connections. Connections can be 
established using a variety of communication protocols (e.g., TCP/IP, IPX, SPX, NetBIOS, 
Ethernet, RS232, or direct asynchronous connections). When connected to the network 50, 
the computer device 10 can function as a Web server that provides access to the matrix 
generator module 20 by other users connected to the network 50 via a client 60. 
[00024] Fig. 2 is a flow diagram of the process of one embodiment of the invention. The 
interface 40 of the computing device 10 receives (step 200) data. The data can be generated 
by, for example, a keyboard (if it is a stand-alone computing device 10 as illustrated in Fig. 
la) or from the client 60 (if connected to a network 50 as illustrated in Fig lb). The data is 
transmitted from the interface 40 to the matrix generator 20, which determines (step 205) 
whether the data relates to a service already performed or a desired service. 
[00025] If the data relates to a service already performed, the matrix generator 20 
determines (step 210) whether a supplier-rating matrix exists for the supplier that performed 
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the service. If a matrix does not exist, the matrix generator 20 generates an initial matrix for 
this supplier. The matrix generator 20 generates (step 215) a job attribute vector with an 
initial value. As described in more detail in the examples below, the service provided by the 
supplier is defined by a plurality of job attributes. Each of these job attributes can have many 
different values. The values of each job attribute are grouped into a plurality of ranges, each 
range referred to as a sub-attribute. The matrix generator 20 defines a dimension in the job 
attribute vector for each sub-attribute. The initial value of each dimension of the vector can 
be, for example, a unity value, such as one. 

[00026] The matrix generator 20 also generates (step 220) a performance vector. As 
described in more detail in the examples below, the matrix generator 20 defines a plurality of 
performance metrics. The matrix generator 20 defines a dimension in performance vector for 
each performance metric. The initial value for each dimension of the performance vector can 
be, for example, an average or middle value, such as 3 out of 5. With the job attribute vector 
and the performance vector defined, the matrix generator 20 generates (step 225) the supplier- 
rating matrix using the two vectors. The examples below describe mathematically combining 
these two vectors to generate the matrix. With the initial matrix created, or if the matrix 
generator 20 determines (step 210) a supplier rating matrix already exists for this supplier, the 
matrix generator 20 proceeds to updating the matrix in response to the received data. 
[00027] Using the received data, the matrix generator 20 generates (step 230) a job 
attribute vector. The job attribute indicates which sub-attributes are applicable for the 
performed service. This can be done, for example, by using ones in those dimensions of the 
vector corresponding to the applicable sub-attributes and using zeros in those dimensions 
corresponding to non-applicable sub-attributes. 
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[00028] The matrix generator 20 also generates (step 235) a performance vector. The 
matrix generator 20 uses the received data to establish a value for each dimension in the 
performance vector. With the job attribute vector and the performance vector defined, the 
matrix generator 20 updates (step 240) the supplier rating matrix using the two vectors 
generated in response to the received data. As described in more detail in the examples 
below, the matrix generator 20 mathematically combines these two vectors generated in 
response to the received data with the existing matrix for that supplier to update the matrix. 
In one embodiment, the matrix generator 20 defines a weighting factor. The weighting factor 
is used to weight the two vectors generated in response to the received data in relation to the 
data in the existing supplier-rating matrix. 

[00029] If the matrix generator determines (step 205) that the received data relates to a 
service that is desired, the matrix generator 20 generates (step 245) a job attribute vector. The 
job attribute indicates which sub-attributes are applicable for the desired service. This can be 
done, for example, by using ones in those dimensions of the vector corresponding to the 
applicable sub-attributes and using zeros in those dimensions corresponding to non-applicable 
sub-attributes. 

[00030] The matrix generator 20 also generates (step 250) a performance vector. The 
matrix generator 20 uses the received data to establish a value for each dimension in the 
performance vector, indicating the desired value and/or the importance of each to the user 
supplying the data. With the job attribute vector and the performance vector defined, the 
matrix generator 20 searches (step 255) the existing supplier rating matrices using the two 
vectors generated in response to the received data. As described in more detail in the 
examples below, the matrix generator 20 mathematically combines these two vectors 
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generated in response to the received data to generate values for an ideal supplier-rating 
matrix. The selector module 30 searches the existing supplier-rating matrices and selects the 
supplier with an existing supplier-rating matrix that is closest to the ideal supplier-rating 
matrix. This selection is returned to the sender of the received data. 

[00031] In another embodiment, not shown, the user who sent the data may be interested in 
a specific supplier. In this embodiment, the selector module 30 compares the supplier-rating 
matrix of the specific supplier with the ideal supplier-rating matrix. The comparison 
identifies the performance metrics that are predicted to exceed the user's desired performance 
for that specific supplier. For example, the value in the existing supplier-rating matrix is 
greater than the corresponding value in the ideal supplier-rating matrix. The comparison also 
identifies the performance metrics that are predicted to not meet the user's desired 
performance for that specific supplier. For example, the value in the existing supplier-rating 
matrix is less than the corresponding value in the ideal supplier-rating matrix. This process 
helps the user identify the future performance of the supplier and assess whether those metrics 
that will not meet the user's desired performance are acceptable. 

[00032] In more detail, one aspect of the invention is a supplier-rating matrix ([SR]). A 
separate supplier-rating matrix is kept for each supplier in the marketplace. A supplier-rating 
matrix contains a summary of a given supplier's past job performance and can be used to 
predict their performance on future jobs. The performance of a supplier is quantified in a 
Performance Vector ( P ). The desired performance (P des ) is specified when looking for a 
supplier and the actual performance (P ac t-i) supplied by the buyer in a post-job questionnaire. 
The attributes of a job ( Ai ) are described mathematically as a vector in a multidimensional 
attribute space. 
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[00033] Updating a Supplier-rating matrix after a job can be described as follows in 
Equation 1 : 

[SR A ]new = f(Paet+ A h [SR A ]old) (1) 

where: 

[SRA]new = Updated Supplier-rating matrix for Supplier A; 
Pact- i ~ Actual performance rating vector for job i; 
Ai = Attributes of job i; and 

[SRA]oid= Supplier-rating matrix for supplier A before updating 
To determine the rating or suitability of a given supplier for a new job (job k) the process is 
run in reverse such that: 

RAk = f(Pdes-h A j} [SR A ]) (2) 
where: 

Raic = Rating for supplier A doing job k; 

Pdes-k= Desired performance rating vector for job k; 

A k = Job Attributes for job k; and 

[SRa] = Supplier-rating matrix for supplier A. 

The variables are more fully described below and in the examples. A simple example 

showing one rating being added to the matrix and using the matrix to predict performance is 

shown in Table 1. A longer example using the same size and type of Supplier-Rating Matrix 

follows. 

Supplier-rating matrix : 

[00034] Each row of the supplier-rating matrix corresponds to a dimension of the 
performance vector, P. These dimensions are metrics by which a buyer could subjectively 
rate a supplier. In the examples, four performance dimensions of, speed, quality, cost and 
service are used. The invention supports the use of more or fewer performance dimensions. 
One of the benefits of the invention is that the number of performance dimensions can be kept 
small to make data collection easier. 
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Table 1: Example 

A Self-Learning Method for Rating Service Providers and Matching them with Prospective Customers 



Part 1: Supplier Rating Matrix Before Feedback 

Avg Score (Range 0-5) Dimensional Tolerance 

.001" .005" .010"+ 

Speed I 2H0 3^00 3^00 

Quality 3.00 3.00 3.00 

Cost 3 00 3.00 3.00 

Service 3.00 3.00 3.00 

Part 2: Applying Customer Feedback 
Actual Performance 
Rating - Pact 

Speed WHIP 
Quality 
Cost 
Service 



Job Attribute Vector, A = 



Turnaround 
0-6 days 1-4 w ks 1 + mon. 



3.00 
3.00 
3.00 
3,00 



3.00 
3.00 
3.00 
3.00 



3.00 
3.00 
3.00 
3.00 



1-10 



Quantity 
11-100 101-1000 



1001+ 



3.00 
3.00 
3.00 
3.00 



3.00 
3.00 
3.00 
3.00 



3.00 
3.00 
3.00 
3.00 



3.00 
3.00 
3.00 
3.00 




— — ■—■ 

Dimensional Tolerance 

001" .005" .010"+ 


Job Attribute* 

Turnaround 
0-6 days 1-4 wks 1+mon. 


Quantity 

1-10 11-100 101-1000 1001+ 


0 0 1 


0 1 , © 


... 0 :: a • . 1 . o 



Vendor Rating Matirx After Applying Feedback 

Filter Value 0.2 





Dimensional Tolerance 
.001" .005" .010"+ 


0-6 


3 days 


Turnaround 
1-4 wks 


1+ mon. 


1-10 




Quantity 
11-100 101-1000 


1001+ 


Speed 


2 80 


3.00 


3.40 




3.00 


3.40 


3.00 




3.00 


3.00 


3.40 


3.00 


Quality 


3.00 


3.00 


3.20 




3.00 


3.20 


3.00 




3.00 


3.00 


3.20 


3.00 


Cost 


3.00 


3.00 


2.60 




3.00 


2.60 


3.00 




3.00 


3.00 


2.60 


3.00 


Service 


3.00 


3.00 


2.80 




3.00 


2.80 


3.00 




3.00 


3.00 


2.80 


3.00 



Part 3: Rating a Supplier for a Proposed Job 
Desired Performance 
Rating: Pdes 

Speed 
Quality 
Cost 
Service 



Job Attribute Vector, A ; 






BflHi 








Dimensional Tolerance 




Turnaround 




Quantity 


.001" .005" .010"+ 


0-6 days 


1-4 wks 1+mon. 


1-10 


11-100 101-1000 1001+ 


0 1 0 


■ ; . ^ 


1 0 




. . . . v ' v 



Vendor Rating for Proposed Job 

Before feedback 



After feedback 



Score 


Raw 


Weighted 


Raw 


Weighted 


Speed 


3.00 


1.25 


3.27 


1.36 


Quality 


3.00 


1 00 


3.13 


1.04 


Cost 


3.00 


0.50 


2.73 


0.46 


Service 


3.00 


0.25 


2.87 


0.24 


Total 




3.00 




3.10 


Average Rating 


3.00 




3.00 





[00035] Columns of the supplier-rating matrix are divided into groups. Each group of 
columns corresponds to a job attribute. An attribute is a measurable characteristic of the job 
such as the quantity of parts made, the turnaround time, the level of precision, or the number 
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of operations required to fabricate a part. Each set of attributes is divided into a number of 
columns with each column assigned a range of attribute values. Any number of sub-columns 
may be employed for each attribute. In one embodiment, 2-5 columns per attribute are used. 
[00036] In the examples, three job attributes are used, dimensional tolerance, turnaround 
time, and quantity. Dimensional tolerance is broken into three columns corresponding to tight 
tolerance (0.001"), medium tolerance (0.005") and loose tolerance (0.010"). These three 
columns represent the three ranges of values of the 'dimensional tolerance' attribute (i.e., the 
sub-attributes). Turnaround time is also divided into three columns, fast (0-6 days), medium 
(1-4 weeks), and long(l month or more). These three columns represent the three ranges of 
values of the 'turnaround time' attribute (i.e., the sub-attributes). The quantity attribute is 
divided into four columns, prototype quantities (1-10), small (11-100), medium (101-1000) 
and large (1001+). These four columns represent the three ranges of values of the 'quantity' 
attribute (i.e., the sub-attributes). For the purpose of illustration, the supplier has achieved a 
3.0/5.0 rating in each category with the exception of the upper left entry. The interpretation of 
the 2.8 rating in the upper left corner of the matrix is "for jobs with a precision of 0.001," this 
supplier gets a "2.8/5.0 rating for Speed." 
Applying Customer Feedback to the Supplier-Rating Matrix 

[00037] In Step 2 of the example in Table 1 , the supplier has just completed a job for the 
customer. The customer rates the supplier in the four areas of speed, quality, cost and service. 
The results are expressed on a scale of 0-5 in the Actual Performance Vector, Pact. In the 
example the vendor earned a perfect 5/5 rating for speed, a good 4/5 rating for quality, poor 
1/5 mark for cost and below average rating of 2/5 for service. Not shown are 3/5, which 
indicates an average rating and 0/5, which indicates an unacceptable rating. 
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Job Attribute Vector 

[00038] The job attribute vector A is created by putting a one in each sub-column that 
corresponds to an attribute of the job. If the attribute does not apply to the job then no entry 
should be made for the specific job. The job being rated in Step 2 of the example has the 
following attributes: loose tolerance of 0.010", turnaround of 1-4 weeks and quantity of 101- 
1000 parts. 

Updating the Supplier-rating matrix 
[00039] Updating is done based on the performance rating given by the buyer, P act , and the 
job attribute, A. There are a variety of formulas that can be used to update the elements of the 
Supplier-rating matrix. In the examples, the following method is used: 



(SR m ,n)new ~ (h)(P a ct-m)(A n ) + (l-h)(SR m;I1 ) 0 id 

where: 

(SR m ,n)new = The new value of the element in row m and column n of the Supplier- 
Rating Matrix; 

h = filter constant (i.e., weighting factor) which is fraction between 0 and 1 . A small 
value weighs past performance stronger; a high (closer to 1) value weighs 
recent performance more strongly. A value of 0.2 is used in the example; 

Pact-m = The 131 th value in the performance vector. For example, for m=2 it is the 2 nd 
element of the Performance vector; 

A n = The n th value in the job attribute vector. For example, for n=2 it is the 2 nd element 
of the Job attribute vector; and 

(SR m , n )oid = The old value of the element in row m and column n of the Supplier-rating 
matrix; 

With reference to the updated matrix in the example, notice that only those columns that 
correspond to attributes of the job that was rated have changed. Thus, a job with certain 
attributes affects only the supplier's ratings for jobs with similar attributes. 
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Using the supplier-rating matrix to evaluate a supplier 

[00040] The supplier-rating matrix can be used to predict a supplier's performance for a 
given job. This process is shown in Step 3 of Table 1. Inputs to the process are: (1) a 
description of the job which in-turn yields a job attribute vector, A, and (2) a description of 
the desired performance as codified in the desired performance vector Pdes. 
[00041] The overall rating, R, is calculated from the weighted average of the Ratings 
associated with each performance metric. The rating for the mth performance metric, Rm, is 
calculated as follows: 

R m = (E(Ai*SR m ,D/(I^) N 



W I — 1 i = ] 



Jl where: 

tH Rm = the rating associated with the m th performance criteria; 

IJ! Ai = the i* element of the job attribute vector; 

n SR m ,i = The element of the supplier-rating matrix associated with the m th row and i th column; 

¥j* and 

y N= total number of columns in the supplier-rating matrix. 

The overall Rating, R, is a weighted average of the performance ratings calculated as follows: 

m -M j =M 

R= (lR m *Pm)/(lPm) 

m - 1 m = I 

where: 

P m = the buyer performance rating for performance criteria m; and 
M = total number of performance criteria 

[00042] In the example a new buyer is looking to evaluate the supplier. The buyer's 
performance criteria are expressed as: high desire for speed (5/5), concerned about quality 
(4/5), not to concerned about cost (2/5) and very little concern about service (1/5). A buyer 
constructs a desired performance vector like this to highlight his need for speed and quality. 
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Not used in this example are desired performance values of (3/5) corresponding to medium 
concern and (0/5) which indicates that the attribute is irrelevant to the buyer. 
[00043] The proposed job has the following attributes: medium tolerance (0.005"), medium 
turnaround (1-4 wks), and medium quantity (101-1000) parts. The resulting rating for the 
supplier is 3.10, which is higher than the rating of 3.0 the supplier would have received based 
on the initial supplier-rating matrix. The increase is attributed to the good job that the 
supplier did on a similar job. The buyer further notes that the supplier's ratings of 3.27 for 
speed and 3.13 for quality indicate better performance in the areas that count most for this 
buyer. 

Using the matrix with Multiple Jobs 

[00044] A second example has been created to show the effect of using the method with 
multiple jobs. In this example a supplier starts with a perfectly average 3/5 rating in each cell 
of the supplier-rating matrix as shown below in Table 2. 

Table 2: Example 

Initial supplier Rating Matrix 





Dimensional Tolerance 




Turnaround 






Quantity 








.001" 


.005" 


.010"+ 


0-6 days 


1-4 wks 


1+ mon. 


1-10 


11-100 101-1000 


1001 + 


Speed 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


Quality 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


Cost 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


Service 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 


3.00 



[00045] The supplier completes Job 1, which calls for loose tolerances, 1-4 week 
turnaround and 1 1-100 parts. The buyer rates the supplier high for speed (5/5) and quality 
(4/5), but is not impressed with the cost (2/5) or service (2/5). The performance vector, job 
attribute vector are shown below in Table 3, which includes the updated supplier-rating 
matrix. 
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Table 3: Example 



Job1: 

Actual Performance 
Rating: Pact 

Speed 
Quality 
Cost 
Service 




st 

high quality 
expensive 
middling service 
loose tolerance 

Dimensional Tolerance 
.001" .005" .010"+ 



■Mi 



med turnaround 



Job 1 Attribute Vector, A= 6 



5 



1 



Turnaround 
0-6 days 1-4 wks 1+mon. i 1-10 



med-high quantity 



Quantity 
11-100 101 




Vendor Rating Matirx After Applying Feedback from Job 1 

Filter Value 0.2 





Dimensional Tolerance 
.001" .005" .010"+ 


0-6 days 


Turnaround 
1-4 wks 


1+ mon. 


1-10 




Quantity 
11-100 101-1000 


1001 + 


Speed 


3.00 


3.00 


3.40 


3.00 


3.40 


3.00 




3.00 


3.40 


3.00 


3.00 


Quality 


3.00 


3.00 


3.20 


3.00 


3.20 


3.00 




3.00 


3.20 


3.00 


3.00 


Cost 


3.00 


3.00 


2.80 


3.00 


2.80 


3.00 




3.00 


2.80 


3.00 


3.00 


Service 


3.00 


3.00 


2.80 


3.00 


2.80 


3.00 




3.00 


2.80 


3.00 


3.00 



[00046] The supplier does a poor job on Job 2. This job calls for tight tolerances, 1 -4 week 
turnaround and 1 1-100 parts. The buyer rates the supplier poorly (2/5) in all areas with an 
especially low mark (1/5) for quality. The performance vector, job attribute vector are shown 
below in Table 4, which includes the updated supplier-rating matrix. 

Table 4: Example 

Job 2: 

Actual Performance 
Rating: Pact 

Speed flKJS Illjfi s,ow 

Quality ^1|fcp° or <l ua,it y 

Cost - - ^ilii 1 ex P ensive 

Service RSl^KHi midling service 

low tolerance medium turnaround low quantity 

Dimensional Tolerance Turnaround '**' * ^ ' '^QuQ^ty ' ^ 

.001" .005" .010"+ 0-6days 1-4 wks 1+ mon. 1-10 11-100 101-1000 1001+ 

Job 2 Attribute Vector, A - 

Vendor Rating Matirx After Applying Feedback from Job 2 

Filter Value 0.2 





Dimensional Tolerance 
.001" .005" .010"+ 


0-6 days 


Turnaround 
1-4 wks 


1+ mon. 


1-10 




Quantity 
11-100 101-1000 


1001 + 


Speed 


2.80 


3.00 


3.40 


3,00 


3.12 


3.00 




3.00 


3.12 


3.00 


3.00 


Quality 


2.60 


3.00 


3.20 


3.00 


2.76 


3.00 




3.00 


2.76 


3.00 


3.00 


Cost 


2.80 


3.00 


2.80 


3.00 


2.64 


3.00 




3.00 


2.64 


3.00 


3.00 


Service 


2.80 


3.00 


2.80 


3.00 


2.64 


3.00 




3.00 


2.64 


3.00 


3.00 



[00047] The supplier completes Job 3 with high customer satisfaction. This job calls for 
loose tolerances, quick turnaround and a prototype number of (1-10) parts. The buyer rates 
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the supplier excellent for speed (5/5) and very good (4/5) in the remaining categories of 



quality, cost and service. The performance vector, job attribute vector are shown below in 
Table 5, which includes the updated supplier-rating matrix. 

Table 5: Example 



Job 3: 

Actual Performance 
Rating: Pact 

Speed 
Quality 
Cost 
Service 



Job 3 Attribute Vector, A = 




fast delivery 
good quality 
expensive 
Imidiling service 
high tolerance quick turn 

Job 



Dimensional Tolerance 
.001" .005" .010"+ 



0 



Turnaround 
0-6 days 1-4 wks 



medium high quanitity 



Quantity 
1-10 11-100 101 




IBlillll 



-1000 1001 + 



ft 



"0 



Vendor Rating Matirx After Applying Feedback from Job 3 

Filter Value 0.2 





Dimensional Tolerance 




Turnaround 








Quantity 








.001" 


.005" 


.010"+ 


0-6 days 


1-4 wks 


1+ mon. 


1-10 




11-100 101-1000 


1001 + 


Speed 


2.80 


3.00 


3.72 


3.40 


3.12 


3.00 




3.40 


3.12 


3.00 


3.00 


Quality 


2.60 


3.00 


3.36 


3.20 


2.76 


3.00 




3.20 


2.76 


3,00 


3.00 


Cost 


2.80 


3.00 


3.04 


3.20 


2.64 


3.00 




3.20 


2.64 


3.00 


3.00 


Service 


2.80 


3.00 


3.04 


3.20 


2.64 


3.00 




3.20 


2.64 


3.00 


3.00 



Applying what has been learned. 



[00048] Two new buyers approach the supplier with jobs. The first buyer, with Job A, has 
a tight tolerance, medium turn-around, and medium quantity job. As illustrated in Table 6, 
the buyer is most interested in speed and quality (5/5), and less concerned about price (2/5) 
and service (3/5). The resultant vendor rating for proposed Job A is an unimpressive 2.86. 
The buyer is especially concerned about the low 2.79 rating for quality. This relatively low 
score is the result of the poor performance of the supplier in Job 2, which also had tight 
tolerances. 
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Table 6: Example 



Proposed Job A 

Desired Performance 
Rating: Pdes 

Speed 
Quality 
Cost 
Service 



speed is important 
l^rrFSSpB quality is important 
I lyiL: - less concern about cost for this part 



medium turn 



Job A Attribute Vector, A = 



tight tolerance 

Dimensional Tolerance 
.001" .005" .010"+ 
1 0 0 



Turnaround 
0-6 days 1 -4 wks 1 + mon. 



medium quantity 

Quantity 

1-10 11-100 101-1000 1001 + 



Vendor Rating for Proposed Job A 

After feedback 
Score Raw Weighted 

Speed 2.97 0.99 

Quality 2.79 0.93 low quality number is cause for concern 

Cost 2.81 0.38 

Service 2,81 0.56 



Total 



2.86 unimpressive rating, look for a different vendor 



[00049] The second buyer, with Job B, has a loose tolerance, quick turn-around, and 
medium quantity job. As illustrated in Table 7, the buyer is most interested in speed and 
quality (5/5), is concerned about price (4/5) and less concerned about service (3/5). The 
resultant vendor rating for proposed Job A is an impressive 3.20. The buyer will be especially 
excited about the high speed rating of 3.37 and the good quality number of 3.19. This 
impressive score is the result of the good performance of the supplier in jobs 1 and 3, which 
more closely resemble this job. 
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Table 7: Example 

Proposed Job B 

Desired Performance 
Rating: Pdes 

Speed s P eed is important 

Quality ^E^^^S^H quality is important 

Cost HBb-^^W cost less cruciai 

Service 

loose toterance -^^A 111 ? 1 !^ e ^^%^ B ^^^^ 

Dimensional Tolerance Turnaround — jbUl-s 

.001 11 .005" .01 0"+ 0-6 days 1-4 wks 1 + mon. 1-10 11-100 101-1000 1001 + 
Job B Attribute Vector, A = IFIiSISll^ 



Vendor Rating for Proposed Job B 



Score Raw Weighted 

Speed 3.37 0.99 does this kind of job fast 

Quality 3.1 9 0.94 and with high quality 

Cost 3.08 0.72 

Service 3.08 0.54 

Total 3.20 impressive rating, this is a good choice 



Additional Features 

[00050] In other embodiments, a supplier-rating matrix can have many more columns 
associated with additional Job attributes. In plastic processing, for example, there could be 
columns associated with the size of press involved, the type of polymer used, the secondary 
operations required, or the type and amount of filler material used in the plastic. 
[00051] The supplier-rating matrix starts with some initial values. In one embodiment, a 
trained quality professional could estimate values for each entry in the supplier-rating matrix 
based on an inspection of the supplier's facility and interviews with past customers. Over 
time the input from actual customers will adjust for any incorrect initial assumptions. 
Essentially, the matrix 'learns' from the experience of the customers. 

[00052] The filter constant (i.e., the weighting factor), h, in the preceding example can be 
modified over time to give more or less weight to recent performance evaluations. For 
example, suppliers with only a few evaluations may benefit from a higher filter constant so 
that a single good rating can move there numbers significantly. On the other hand, suppliers 
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who have received several evaluations may benefit from a lower h value giving them a more 
stable rating. 

[00053] Individual elements of the supplier-rating matrix can also be used to predict 
performance if a potential buyer has a narrow focus. For example, a buyer concerned with 
tight tolerance jobs could compare ratings from the tight tolerance column of the supplier- 
rating matrix independent of other attributes. 
Equivalents 

[00054] The invention can be embodied in other specific forms without departing from the 
spirit or essential characteristics thereof. The foregoing embodiments are therefore to be 
considered in all respects illustrative rather than limiting on the invention described herein. 
Scope of the invention is thus indicated by the appended claims rather than by the foregoing 
description, and all changes which come within the meaning and range of equivalency of the 
claims are therefore intended to be embraced therein. 
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